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understanding  the  quality  role  in  relation  to  specific  contractual  language. 
It  is  also  designed  to  illustrate  that  product  quality  is  an  all  encompassing 
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The  purpose  of  this  guidance  document  is  to  aid  those  USAF 
R&D  managers,  contracting  officers,  and  acquisition  managers  who  are 
involved  in  the  procurement  of  defense  material  by  providing  information 
that  will  help  assure  that  the  products  being  acquired  will  satisfy 
the  requirements  of  those  who  will  use  the  equipment. 

It  is  intended  to  be  an  adjunct  to  other  existing  policies 
and  procedures  and  will  provide  a  conceptual  framework  and  basis  for 
understanding  the  quality  role  in  relation  to  specific  contractual 
language.  It  is  also  designed  to  illustrate  that  product  quality  is 
an  all  encompassing  concern  and  numerous  organizational  elements 
contribute  to  overall  product  quality.  Quality  is,  in  fact,  everyone's 
business . 
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I.  OVERVIEW 


Acqusition  managers,  as  well  as  R&D  managers  and  contracting 
officers,  must  consider  quality  in  terms  of  all  of  the  product's  attributes 
as  related  to  the  needs  of  the  user.  Their  view  of  product  quality  must 
not  be  inhibited  by  any  arbitrary  organizational  division  or  "special" 
product  features.  The  manager  must  consciously  recognize  product  quality 
as  a  primary  responsibility;  equipment  delivered  on  schedule  at  the  agreed 
cost  will  be  of  little  benefit  if  it  does  not  meet  the  users  needs.  The 
manager  must  acquire  a  product  of  specified  excellence  at  the  least  possible 
cost  in  the  public  market  place.  These  responsibilities  are  the  managers 
alone  and  it  necessitates  specific  responsibilities  assigned  to  specific 
individuals  who  must  be  competent  in  whatever  technological  and  administrativ 
disciplines  apply. 

Product  quality  is  of  interest  at  the  highest  levels  of  government 
and  it  is  essential  to  the  national  interest.  It  is  a  real,  not  an  abstract, 
consideration  for  those  personnel  whose  lives  may  depend  on  the  successful 
operation  of  a  weapon  or  other  type  of  system.  Product  quality  must  begin 
with  those  individuals  who  will  use  the  product;  it  is  their  responsibility 
to  define  in  exacting  detail  what  characteristics  are  essential  to  their 
mission.  If  no  such  definition  is  provided,  it  will  be  a  matter  of  chance 
if  their  requirements  are  met.  Furthermore,  failure  of  the  product  to 
perform  other  missions  than  described  by  the  user  reflect  not  on  the  producer 
or  the  product,  but  on  the  using  organization  who  did  not  convey  all  of  the 
necessary  requirements  or  characteristics  needed. 

The  Office  of  Management  and  Budget  issued  the  0MB  Circular  A-109, 
which  is  concerned  with  the  system  acquisition  process  in  all  of  the  govern¬ 
mental  departments.  The  circular  essentially  results  in  the  establishment 
of  national  priorities  for  major  systems  of  all  kinds.  It  is  very  limited 
in  terms  of  specific  performance  concepts,  dealing  instead  with  broad  system 
mission  purpose  and  concepts.  Product  quality  at  this  stage  is  described 
only  in  the  broadest  general  terms. 
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At  the  Department  of  Defense  (DoD)  level,  quality  begins  to  appear 
from  two  different  perspectives.  The  first  is  in  the  5000  series  of  DoD 
Directives;  principally  5000,2  where  specific  performance  requirements  must 
be  defined  in  Annex  A.  The  first  quality  milestone,  that  identifies  the 
user's  needs  in  broad  operational /performance  terms,  is  established  at  this 
point.  The  product  quality  characteristics  are  normally  stated  in  dynamic 
terms,  i.e.,  performance.  They  may  include  static  characteristics  as  well, 
e.g.,  overall  weight  or  dimensions. 

Product  quality  characteristics  that  are  dynamic  are  concerned 
with  operating  parameters  or  functions.  Examples  of  these  operating  or 
dynamic  functions  may  include  speed,  horsepower,  RF  power,  bandwidth,  etc. 
These  dynamic  characteristics  may  be  evaluated  for  conformance  in  instant 
tests  such  as  maximum  power  thrust  or  they  may  also  be  evaluated  in  time 
related,  i.e.,  reliability  tests  which  require  continuing  performance 
without  failure  under  some  prescribed  conditions  for  some  period  of  time. 
Static  or  physical  characteristics  are  measured  on  an  instant  basis  for 
conformance.  All  such  measurements,  tests,  analysis,  demonstrations,  etc. 
are  measures  of  product  quality  and  conformance  to  them  may  require 
specialized  technical  discipline  capabilities  such  as  gamma  rays,  ultrasonics, 
liquid  crystal  inspection,  etc. 

After  the  requirements  have  been  described,  the  responsibility 
issues  becomes  that  of  (1)  who  must  assure  the  equipment  will  be  capable 
of  attaining  the  necessary  quality  and  (2)  who  will  measure  the  equipment 
to  assure  conformance  to  those  quality  requirements.  In  organizational 
terms,  the  activity  responsible  for  technical  requirements  or  technical 
activity,  must  define  the  technical  characteristics  of  the  equipment  and 
prepare  necessary  product  descriptions  in  terms  of  drawings,  specifications 
and  other  forms  of  documentation.  This  technical  activity  must  not  only 
define  the  product's  technical  characteristics  but  must  also  be  responsible 
for  prescribing  "inspection  testing  or  other  contract  quality  requirements 
that  are  essential  to  assure  the  integrity  of  products  and  services," 

(DAR  14-201).  The  technical  activity  then  has  a  dual  quality  role: 
assuring  quality  of  design  conformance  to  user  need;  and  establishment  of 
methodology  for  determing  compliance  to  design. 
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Definition  of  requirements  in  a  contractual  sense  may  begin  in 
the  conceptual  phase  of  the  life  cycle  and  will  usually  be  limited  to 
minimal  technical  objectives  that  are  anticipated  to  be  achievable  within 
the  contract  time  constraints.  Operational  environments,  safety  factors 
and  interface  considerations  with  other  systems  may  be  involved.  By  the 
conclusion  of  this  phase,  elements  of  design  requirements  and  methods 
necessary  to  assure  they  have  been  achieved  will  have  been  performed.  At 
the  validation  phase,  the  design  quality  of  performance  requirements  and 
the  quality  methodology  for  assuring  they  have  been  achieved  are  subject 
to  better  definition  and  configuration  perspectives.  Management  controls 
in  some  form  are  necessary. 

By  the  full-scale  engineering  development  phase,  the  quality  of 
design  in  terms  of  equipment  characteristics  should  have  been  established. 
Measures  for  assuring  that  design  requirements  have  been  accomplished  should 
also  have  been  developed.  The  quality  of  design  in  terms  of  its  own 
excellence  then  becomes  an  issue.  It  is  not  enough  that  required  peformance 
can  be  achieved,  but  design  reviews  must  be  performed  which  relate  to  such 
matters  as:  tolerances,  safety  factors,  part  deratings,  part  selection 
criteria,  ageing  considerations,  worst  case  stress  analysis,  transients 
expected,  redundancy,  sneak  circuit  analysis,  etc.  Excellence  of  quality 
of  design  is  a  vital  quality  function  which  the  acquisition  manager  must 
assure  is  confirmed  by  engineering  elements  of  the  organization.  Appendix  1 
provides  a  typical  design  review  procedure  which  will  assist  in  providing 
assurance  that  the  quality  of  design  has  been  considered.  Figure  1-1 
depicts  cypical  organizations  that  participate  in  the  design  review 
process. 


Also,  by  the  full-scale  engineering  development  phase,  the 
methods  of  assuring  that  the  equipment  conforms  to  the  established  design 
requirements  must  be  accomplished.  This  begins  with  assuring  that  a  system 
definition,  usually  in  the  form  of  a  specification  or  series  of  specifications, 
is  prepared  which  describes  all  characteristics  including  physical  and 
performance  characteristics.  If  reliability  requirements  are  established, 
they  must  be  stated  numerically  with  appropriate  performance  levels  in  terms 
of  mean  time  before  failure  (MTBF),  or  probability  of  mission  success. 
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Figure  1-1 


Participation  in  the  Design  Review  Process 


In  addition,  demonstration  methodology  must  be  identified  which  will  assure 
the  requirements  have  been  accomplished  (RIL-STD  781).  For  spot  or  instant 
performance  demonstrations,  procedures  must  be  established  to  preclude 
disagreements  as  to  whether  or  not  the  requirement  has  been  demonstrated. 
Physical  requirements  must  also  be  described.  If  maintainability  is  a 
requirement,  it  should  be  stated  in  terms  of  mean  time  to  repair  (MTTR)  or 
maintenance  man  hour  per  operational  hour.  Demonstration  methodology  must 
also  be  defined  (MIL-STD  471).  Quality  of  design  may  also  become  a  maintain¬ 
ability  issue  as  in  requirements  for  accessabil ity  or  test  points  or  other 
ease  of  maintenance  considerations  which  may  be  specified. 


In  addition  to  the  standardization  and  otherwise  specified 
procedures  for  testing  which  commonly  appear  in  specifications,  specialists 
should  examine  tests,  inspections  and  measurements  to  be  performed  and 
consider  the  need  for,  and,  if  necessary,  develop  other  structured  inspection/ 
test  approaches.  Consideration  must  be  given  to  the  need  for:  pre-production 
samples,  periodic  production  samples,  pilot  tests,  methods  for  lot  formation 
or  sample  selection,  testing  outlines  including  sequence,  characteristics 
data,  etc.  Ultimately,  the  specification  must  assure  the  product  conforms 
to  the  requirements  by  providing  physical  characteristics,  including  work¬ 
manship  by  a  measurement  or  examination,  and  dynamic  characteristics  by 
either  instant  or  time  related,  e.g.,  reliability  tests  with  defined  test 
and  inspection  methodology.  MIL-STD  490  provides  a  standard  PoO  method  for 
specifying  practices  for  preparing  specifications. 

It  can  be  seen  that  quality  is  a  very  broad  subject  which  interfaces 
with  many  organizations,  and  that  quality  considerations  transcend  traditional 
organizational  boundaries.  Aside  from  all  of  the  specific  technical  require¬ 
ments,  the  management  systems,  the  contractor  performance  evaluation  and  the 
other  myriad  problems  the  project  manager  faces,  motivation  is  also  a  crucial  issu 

Profit  is  important  and  essential  to  the  continuing  existence 
of  any  firm  and,  in  theory,  profit  will  equate  to  quality.  In  the  arena 
of  military  systems  procurement,  however,  it  is  not  as  clear  cut  an  issue 
as,  for  example,  when  the  commercial  air  carrier  purchases  major  systems. 

Although  it  is  not  necessarily  logical,  the  inoperative  commercial  aircraft 
seems  to  be  more  important  to  the  producer  than  inoperative  military 
systems.  In  the  commercial  market,  the  feedback  is  faster  and  producer 
and  purchaser  work  together  in  concert  to  solve  the  problem. 

Based  on  interviews  and  existing  policy  reviews,  there  appears 
to  be  an  increasing  dysfunctional  phenomenon  in  the  firm's  motivational 
inspiration  as  the  system  moves  through  the  life  cycle.  Motivation  is 
high  through  the  full-scale  development  phase  and  into  early  production. 

Through  the  later  life  cycle  phases,  the  project  manager  must  somehow 
provide  government  motivation  that  the  firm  will  respond  to.  Mo  ready 
solution  can  be  presented  due  to  the  different  conditions  that  exist  on 
each  system.  The  project  manager  must  devote  attention  to  producer  quality 


measurements  on  a  continuing  basis  and  must  develop  solutions  for  each  case. 
Solutions  range  from  modifying  contracts  to  include  incentives,  to  encouraging 
the  development  of  contractor  motivational  programs,  to  creating  psychological 
rewards  such  as  developing  a  continuing  dialogue  with  contractor  personnel 
at  the  junior  executive  and  operator  levels.  Early  NASA  motivational  programs 
which  included  introducing  astronauts  to  employees  at  all  levels  is  an 
example  of  an  imaginative,  innovational ,  essentially  no  cost,  program  which 
produced  substantial  quality  benefits. 

Figure  1-2  graphically  illustrates  areas  where  it  is  necessary 
to  encourage  further  contractor  motivation  over  the  system  life  cycle. 

Figure  1-2 

Life  Cycle  Motivational  Dysfunction 


Contractor  Motivation 

Government  Motivational  Emphasis  Needed 
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II.  QUALITY  FUNCTIONS  IN  THE  LIFE  CYCLE 

The  preceding  section  of  the  guide  was  designed  as  a  broad 
oversight  of  quality  considerations  involved  in  the  system  acquisition 
process.  This  section  will  more  closely  relate  to  duties  and  responsibilities 
normally  assigned  to  the  Quality  Assurance  and  Reliability  Organizations  in 
the  various  life  cycle  phases. 

A.  CONCEPTUAL  PHASE 

Consideration  must  be  given  to  the  user  needs  as  stated  in  the 
Mission  Element  Needs  Statement  (HENS)  in  relation  to  what  appears  in  the 
system  technical  description.  The  MENS  may  relate  to  deficiencies  in 
existing  capability  in  system  quality  terms  such  as  inadequacies  in 
system  readiness,  mission  performance  or  logistic  support  requirements. 

Quality  planning  at  this  stage  is  essentially  limited  to  establishment  of 
broad  performance  objectives.  Consideration  may  also  be  given  to  the 
state-of-the-art  questions  and  very  preliminary  life  cycle  cost  studies. 

B.  DEMONSTRATION  AND  VALIDATION  (D&V)  PHASE 

Development  of  a  specific  quality  strategy  must  begin  as  soon  as 
possible  after  milestone  0  as  a  part  of  the  overall  system  strategy. 

The  quality  strategy  is  to  a  large  degree,  dependent  upon  the  overall 
acquisition  strategy.  Consideration  can  be  given  to  product  quality,  i.e., 
principally  physical  and  dynamic  requirements.  Bands  of  uncertainty  regarding 
expected  performar.ee  should  be  developed  predicated  on  similar  systems  and 
design  estimates.  Subsystem  performance  achievements  can  be  documented  in 
addition  to  the  principal  technical  objectives  in  the  DCP  Annex  A.  Some 
limited  configuration  controls  must  be  in  effect  so  that  test  results  can 
be  correlated  with  changing  designs  or  test  method.  General  types  of  testing 
that  must  be  performed  can  be  identified  and  test  procedures  standardized 
per  existing  test  procedures,  e.g.,  vibration  tests,  performed  per  method 
201A,  MIL-STD  202. 


A  system  specification  should  be  developed  by  the  end  of  the  DAV 
phase  with  accompanying  performance  requirements.  While  these  requirements 


are  subject  to  tradeoff  against  cost  schedule  and  supportabil ity,  their 
establishment  will  provide  a  measurable  and  definable  performance  baseline. 

The  system  specification  need  only  reflect  primary  subsystem  goals  and  does 
not  necessarily  include  individual  configuration  items.  However,  prime  item 
or  system  specifications  should  be  prepared.  The  system  specification 
should  include  testing  and  inspection  requirements  for:  verification  of 
performance  requirements,  verification  of  physical  characteristics,  R&M 
requirements  demonstrations,  and  such  other  tests/inspections  that  are  required 
to  confirm  that  all  established  requirements  have  been  accomplished.  To 
the  extent  subsystem  speci fications  are  prepared,  the  same  general  guidelines 
prevail.  In  these  specifications,  however,  in  addition  to  definition  of 
functional  and  physical  requirements,  there  must  be  adequate  definitions  of 
interface  parameters  and  complete  test  and  inspection  requirements  necessary 
to  demonstrate  each  specified  functional  characteristic.  Contingency  plans 
should  also  be  developed  for  tailoring  in-plant  management  systems  such  as 
MIL-Q-9358  quality  program  or  MIL-STD-785,  Reliability  Program.  Contractor 
preparation  of  equipment  specifications  must  be  consistent  with  the 
instructions  in  MIL-STD  490  or  MIL-STD  483,  Configuration  Management 
Practices.  At  the  conclusion  of  the  D&V  phase,  it  will  be  necessary  to 
perform  an  independent  quality  assessment  in  accordance  with  AFR  7*-l . 

C.  FULL-SCALE  ENGINEERING  DEVELOPMENT 

During  this  segment  of  the  System  Development  Cycle,  a  second  DoD 
level  review  of  the  system  will  be  performed.  At  milestone  2,  the  Annex  A 
goals  should  be  verified  as  being  achievable  and  threshold  values  specified 
should  be  the  minimum  values  acceptable.  Adequate  investigation  must  be 
performed  to  confirm  that  Government  Funished  Equipment  (GFE)  which  is  to 
be  incorporated  will  also  meet  performance  requirements.  Necessary  liaison 
must  be  established  with  'the  government  activity  furnishing  the  GFE,  e.g., 
appropriate  Air  Logistic  Activity,  to  obtain  and  provide  necessary  product 
information. 

In  support  of  DCP  Annex  A,  a  continuing  summary  of  major  assembly 
performed  test  results  should  be  maintained.  An  ongoing  analysis  and  periodic 
(monthly)  review  of  this  data  will  give  advance  indication  of  quality  problems 
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that  are  predictable  in  required  reliability  demonstrations.  Repeated 
failures  indicate  that  corrective  action  is  ineffective  and  even  if 
statistics  permit  reliability  demonstration  success,  field  performance 
problems  are  likely.  Any  such  data  should  be  prepared  so  that  trends 
are  apparent.  It  may  be  desirable  to  establish  a  routine  meeting  of  the 
government  and  contracting,  engineering,  quality  assurance,  manufacturing 
and  R&M  personnel  to  review  the  data.  Any  such  review,  formal  or  informal, 
should  also  include  Initial  Operative  Test  and  Evaluation  (I0T8E)  using 
command  representatives.  In  the  event  these  informal  exchanges  of 
information  are  formalized,  it  may  be  desirable  to  combine  them  with  existing 
technical  reviews,  or  if  none  are  established,  to  organize  a  formal  joint 
reliability,  maintainability  and  system  quality  review  team. 

During  this  phase,  engineering  reviews  should  be  conducted  on 
an  organized  and  scheduled  basis.  As  mentioned  earlier,  Figure  1-1 
illustrates  typical  technical  disciplines  that  participate  in  design 
reviews  and  Appendix  1  contains  a  typical  approach  to  design  review. 

This  appendix  provides  an  identification  of  the  supporting  material  which 
should  be  available  and  a  list  of  typical  design  issues. 

At  the  same  time,  the  quality  assurance  elements  should  be  devoting 
particular  attention  to  specified  requirements  and  recording  achievements 
in  an  organized  manner.  There  are  existing  requirements  for  recording  and 
monitoring  specified  goals  for  reliability  and  maintainability  requirements . 
This  does  not  obviate  the  need  for  a  logical  organized  approach  to  collecting 
and  recording  specific  product  performance  as  it  relates  to  requirements 
which  do  not  have  associated  reliability  requirements  established,  or  the 
need  for  a  comprehensive  performance  summary.  The  development  and  maintenance 
of  product  performance  history  as  experienced  in  the  development  progress 
will  enable  the  acquisition  manager  to  have  an  overall  perspective  of  the 
quality  of  the  system  as  it  is  developing.  This  data  will,  as  well,  supple¬ 
ment  contractually  required  performance  data  such  as  reliability.  This 
data  will  also  be  applicable  to  the  independent  product  assessment  required 
at  the  end  of  the  full-scale  engineering  development  phase.  Appendix  2 
illustrates  one  method  of  assembling  system  quality  data  to  supplement  other 
information. 

9 


-  **v  * 


0.  PRODUCTION 


Prior  to  a  decision  to  enter  the  production  phase,  the  third 
milestone  review  will  be  held.  The  DCP  will  still  be  the  primary 
documentation  for  use  by  the  Defense  Systems  Acquisition  Review  Council 
(DSARC).  It  will  summarize  the  program  achievements  in  relation  to  the 
requirements  and  overall  acquisition  strategy.  A  primary  consideration 
will  be  the  Annex  A  data  with  current  system  performance  data.  Back-up 
performance  data  as  outlined  in  Appendix  2  and  the  Independent  Product 
Assessment  will  provide  supplementary  or  back-up  data  for  Annex  A  and  the 
Milestone  Reference  File. 

After  award  of  the  production  contract,  the  quality  role  will  be 
limited  essentially  to  assuring  conformance  to  established  requirements. 

While  there  are  numerous  associated  activities,  product  conformance  to 
requirements  is  the  primary  concern.  The  USAF  quality  assurance  organization 
performs  or  participates  in  a  variety  of  non-quality  functions  due  to  their 
presence  in  the  manufacturing  facility  in  substantial  numbers.  These  functions, 
while  not  directly  concerned  with  inspection  and  test,  also  influence  the 
ultimate  success  or  failure  of  contract  objectives.  Despite  the  numerous 
functions  assigned  and  areas  of  involvement,  the  quality  function  essentially 
remains  as  that  of  assuring  that  the  product  conforms  to  contract  require¬ 
ments.  This  determination  is  in  consonance  with  the  basic  DoD  quality 
assurance  philosophy  contained  in  Section  14  of  the  DAR  and  is  essentially 
performed  by  controlling  the  overall  production  process  and  supplemented  by 
such  product  inspection  as  is  required.  This  divides,  for  all  practical 
purposes,  the  USAF  in-plant  quality  assurance  approach. into  two  areas:  (1) 
control  of  production  processes,  and  {?.)  direct  product  inspection.  The 
central  quality  assurance  responsibility  still  remains  that  of  confirming 
that  the  system  conforms  with  specified  requirements  and  taking  title  to  it 
for  the  government  by  signifying  on  the  Material  Inspection  and  Receiving 
Report,  that  it  does  meet  requirements. 


III.  QUALITY  INTERFACES  IN  THE  DEFENSE  ACQUISITION  REGULATION 


The  preceding  paragraphs  of  the  guide  have  dealt  with  the  quality 
role  in  post  contractual  terms.  Section  14*  of  the  DAR  (Procurement  Quality 
Assurance)  is  concerned  with  all  phases  of  the  life  cycle  and  deals  with  the 
post  contractual  situation  as  well.  There  are  several  areas  in  other  sections 
of  the  DAR  where  product  qual i ty/ performance  is  a  consideration.  The  purpose 
of  this  section  of  the  guide  is  to  identify  those  DAR  interfaces  where 
product  quality  should  be  a  consideration. 

DAR  1-319  which  is  concerned  with  renegotiation  practices  requires 
the  review  of  profit  of  defense  contractors  performing  renegotiable  contracts 
and  subcontracts.  The  first  item  of  the  report  concerning  performance  is 
"the  product  exceeded  met  or  fell  below  the  contract  requirements."  This  is 
a  clear  requirement  for  product  quality  data.  To  be  responsive  to  the 
requirements  requires  a  rational,  consistent  analysis  of  product  quality. 

It  would  be  an  error  to  discount  this  opportunity  on  the  grounds  that,  if 
accepted,  the  product  met  requirements.  This  is  a  direct  opportunity  to 
relate  product  quality  to  profit,  consistent  with  the  general  DAR  guidance. 

Component  Breakout  (DAR  1-326)  is  an  activity  that  may  cause 
product  performance  deficiencies.  The  two  elements  of  DoD  policy  on  whether 
or  not  to  breakout  a  component  are:  (1)  cost  savings,  and  (2)  quality, 
reliability,  performance  or  delivery.  There  is  an  obvious  need  for  quality 
considerations  in  the  component  breakout  decision  process. 

Determination  of  the  responsibility  of  prospective  contractors 
(DAR  1-900)  has  traditionally  been  a  part  of  the  quality  role.  The  primary 
determination  is  made  by  the  contracting  officer  with  the  assistance  of 
numerous  functional  areas  including  quality  assurance.  It  strengthens  t.iese 
proceedings  when  contract  administration  elements  routinely  prepare  assess¬ 
ments  of  contractor  performance  on  previous  contracts.  This  is  not  normally 
an  additional  manpower  requirement  since  most  defense  contractors  receive 
successive  contracts  and  pre-award  surveys  are  frequent  occurences.  Such 
data  collected  as  depicted  in  Appendix  2  would  be  of  material  assistance. 

It  would  also  provide  a  rational  data  base  for  post-award  orientation 
(DAR  1-1800). 
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Procurement  planning  1-2100  is  an  area  where  the  quality  role 
is  essential  as  a  part  of  realistic  judgments.  The  sample  procurement 
plan  format  in  OAR  1-2102  has  several  areas  where  quality  considerations 
have  merit.  Examples  are  Integrated  Logistics  Support  Planning  Concept, 
Application  of  Life  Cycle  Cost,  R&M  and  Government  Furnished  Material.  The 
quality  assurance  element  should  either  expand  the  above  paragraph  with  a 
quality  section  or  prepare  a  separate  paragraph. 

Profit,  DAf!  3-401  (b),  is  identified  here  as  a  basic  industrial 
objective.  It  is  to  be  directly  related  to  stimulating  contractor  perfor¬ 
mance  by  rewarding  effective  performance  with  high  profits,  mediocre 
performance  with  mediocre  profits  and  poor  performance  with  poor  profits. 
The  principal  contract  objective  is  to  obtain  a  quality  product  delivered 
on  schedule  at  a  reasonable  price.  It  follows  then  that  performance  in 
terms  of  product  quality  should  always  be  considered  when  profit  related 
determinations  are  made.  This,  in  turn,  requires  that  some  standard  method 
of  measuring  quality  be  developed  and  applied  uniformly  to  assure  that 
product  quality  is  integrated  into  an  overall  motivational  approach. 

The  Cost  Plus  Award  Fee  (CPAF)  concept  discussed  in  DAR  3-405.5 
allows  profit  to  be  related  to  areas  of  performance  where  measures  of 
progress  or  success  are  less  susceptible  to  finite  measurements.  In  this 
case,  "quality"  is  specifically  mentioned.  Fees  that  are  available  in 
either  case  are  subject  to  limitations  as  described  in  DAR  3-405. 6(c)(2) . 

In  addition,  it  is  likely  that  quality  will  be  only  one  of  the  incentive 
areas  and  the  interrelationship  of  the  fee  areas  must  be  evaluated 
carefully. 


Contracts  with  performance  incentives,  DAR  3-407.2,  can  be 
considered  as  a  profit  motivational  contract  that  has  the  potential  for 
improving  product  quality  through  the  quality  management  system.  Performance 
as  used  in  this  paragraph  is  not  limited  to  technical  performance  of  the 
product  but  relates  to  the  performance  of  the  contractor  as  well.  As  an 
example,  a  plan  could  be  designed  which  would  relate  to  end  item  inspection/ 
test  findings.  A  clever,  innovative  product  quality  program  interfacing 
with  a  profit  relationship  could  be  designed  which  would  utilize  the 
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profit  motive  to  improve  production  techniques.  Such  a  program  would  aid 
in  reducing  the  undiscovered  defects  which  are  frequently  present  due  to 
the  normally  acceptable  defect  risk  levels.  R&M  requirements  are  an  obvious 
application  of  this  contracting  mode.  Cne  significant  difference  between 
Performance  Incentives  and  CPAF  Contracts  is  that  a  target  fee  is  established 
about  which  the  fee  can  vary  up  or  down  as  opposed  to  only  positive  fee 
earnings  on  the  CPAF  contract.  If  a  quality  oriented  performance  incentive 
were  to  be  utilized,  it  would  be  necessary  to  specify  exactly  what  the  target 
objective  is  in  measurable  terms,  e.g.,  no  discrepancies  on  50  percent  of 
the  products  at  final  acceptance  inspection. 

The  "Make  or  Buy"  program  described  in  DAR  3-900  and  Clause 
7-204. 20(a)  provides  an  opportunity  to  influence  product  quality.  Make  or 
buy  reviews  should  evaluate  price,  delivery,  quality  and  performance.  The 
contractor  has  the  responsibility  for  make  or  buy  decisions.  These  decisions 
are  accepted  unless  they  are  adverse  to  the  government's  interests. 

Government  evaluation  may  be  simply  that  of  assuring  proper  categorization. 
Cost  is  a  primary  consideration.  However,  quality  considerations  should 
not  be  overlooked. 

DAR  3-800  is  concerned  with  price  negotiation  policies  and 
techniques  applicable  to  prime  and  selected  subcontracts.  There  is  a 
quality  responsibility  role  in  the  contracting  office  and  in  the  Contract 
Administration  Office  to  support  the  contracting  officer.  While  this 
paragraph  of  the  DAR  is  primarily  concerned  with  pricing  opportunities,  it 
provides  a  forum  for  introducing  product  quality  information.  The  weighted 
guidelines  profit  factors  analysis  is  silent  on  product  quality  but  its 
consideration  is  consistent  with  DAR  3-401 (b)  which  associates  profit  with 
performance,  including  reliability,  quality  and  performance. 

The  Leader  Company  method  of  procurement  described  in  DAR  4-701 
requires  special  effort  on  the  part  of  quality  and  production  personnel. 

The  principal  element  of  this  procedure  is  that  the  leader  company  furnishes 
manufacturing  assistance  to  a  second  follower  company  that  is  expected  to 
become  another  supplier  for  the  item/system.  Government  personnel  in  the 
leader  and  in  the  company  plants  should  maintain  a  close  liaison  under  this 
type  of  contracting. 
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Management  systems  described  in  DAR  1-331  are  those  documented 
methods  for  defining  or  stating  policy,  objectives  or  requirements.  Examples 
are  MIL-Q-9858  Quality  Program,  MIL-STD  785  Reliability  Program,  MIL-STD  470 
Maintainability  Program  Requirements  and  MIL-STD  483  Configuration  Management 
Practices.  The  foregoing  management  systems  are  particularly  relative  to 
product  quality.  They  all  require  the  contracting  firm  to  perform  some 
management  services  in  some  prescribed  fashion.  All  contribute  to  product 
quality  and  they  are,  in  effect,  service  contracts  within  the  overall 
contract  framework  to  be  performed.  They  are  all  of  Interest  to  the 
acquisition  manager  and  should  be  reviewed  by  the  manager  periodically. 

Much  of  this  document  has  dealt  with  specifics  which  the 
acquisition  manager  may  not  have  to  be  concerned  with  personally.  This 
guide,  though,  was  designed  to  provide,  in  addition,  an  overall  perspective 
that  quality  is  everyone's  business  and  quality  functions  are  performed  by 
practically  everyone  involved  in  contract  administration.  The  quality 
assurance  organization  can,  due  to  its  unique  involvement  in  many  of  the 
firm's  activities,  provide  an  effective  overview  for  assimilating  and 
providing  the  contract  manager  with  product  quality  information. 


14 


IV.  CONTRACT  LANGUAGE  AND  PRODUCT  QUALITY 


This  part  of  the  guide  is  to  provide  insight  and  information 
relative  to  significant  clauses  that  are  frequently  included  in  contracts 
and  which  can  have  an  impact  on  product  quality.  A  listing  of  such  clauses 
is  contained  in  Appendix  3. 

The  principal  contract  objective  is  to  obtain  a  quality  product 
delivered  on  schedule  and  at  a  reasonable  price.  In  view  of  the  magnitude 
and  complexity  of  the  overall  contracting  endeavor,  standardization,  as 
far  as  possible,  is  achieved  by  standard  contract  language  contained  in 
the  DAR.  While  these  contractual  terms  and  conditions  are  highly  structured 
and  stylized,  they,  nevertheless,  are  closely  related  to  successful  perfor¬ 
mance  of  the  quality  function.  The  DAR  is  clear  as  to  when  these  clauses 
must  be  included  in  contracts  and  when  they  are  optional.  Some  clauses 
such  as  the  Quality  Program,  are  well  known  and  therefore,  will  not  be 
discussed.  Those  that  follow  are  considered  to  be  worthy  of  inclusion  in 
the  guidance  document.  The  selection  and  application  of  inspection/quality 
clauses  are  described  in  detail  in  DAR  14-101  and  are  summarily  treated  here. 

A.  STANDARD  QUALITY  CLAUSES 

Quality  Program,  DAR  7-104.28 

This  clause  may  be  used  in  all  supply  and  R&D  contracts.  It 
incorporates  MIL-Q-8958A  in  the  contract. 

Inspection  System,  DAR  7-104.33 

This  clause  may  be  used  in  all  supply  contracts.  It  incorporates 
MI L- I -45208  in  the  contract. 

Inspection,  DAR  7-103.5 

This  clause  is  required  in  all  fixed  price  supply  and  R&D  contracts 
with  a  value  of  over  $10,000.  This  clause  requires  that  the  contractor 
maintain  an  inspection  system  responsive  to  the  products  being  procured  and 
that  it  be  acceptable  to  the  government  and  also  that  inspection  records 
be  kept  at  least  for  the  duration  of  the  contract.  This  is  a  powerful  contract 


requirement  despite  its  deceptive  simplicity.  It  further  requires  that  the 
contractor  provide  "all  reasonable  facilities  and  assistance"  for  the  safety 
and  convenience  of  government  inspectors  in  the  performance  of  their  duties. 

In  the  event  the  contractor  requests  government  inspection  and  the  supplies 
are  not  ready  for  inspection,  or  in  the  event  they  are  rejected  and  subsequent 
government  inspections  are  required,  the  contractor  may  be  charged  for  the 
extra  government  efforts.  DAR  14-403(e)  and  AFCMDR  74-1  elaborates  on 
government  actions  in  this  regard.  This  portion  of  the  clause  is,  in 
effect,  a  penalty-based  motivational  device  and  it  should  not  be  construed 
as  an  end  in  itself  but  simply  as  one  form  of  motivation  available  to  the 
quality  function. 

Responsibility  for  Inspection,  PAR  7-103.24 

This  clause  may  be  used  as  the  only  contractor  quality  management 
requirement  required  in  Small  Purchase  Fixed  Price  Supply  contracts  with  a 
value  of  $10,000  or  less.  It  is  discussed  in  DAR  14-101.1  and  its  effect 
is  to  make  the  contractor  responsible  for  the  performance  of  inspections 
and  tests  which  are  identified  in  the  contract  specifications  and  drawings. 

It  also  incorporates  a  requirement  for  verifying  techical  requirements  as 
described  in  commercial  items  described  by  a  manufacturer 's  part  number. 

B.  CLAUSES  DIRECTLY  RELATED  TO  PRODUCT  QUALITY 

First  Article  Tests,  DAR  7-104.55 

First  Article  Tests  are  intended  to  be  a  demonstration  that  a 
firm  can  provide  a  product  meeting  all  requirements.  The  test  is  particularly 
desirable  when: 

o  the  contractor  has  not  furnished  the  item 
previously; 

o  substantial  changes  have  been  made  or  production 
has  been  discontinued  for  an  extensive  period; 

o  the  article  is  described  only  by  a  performance 
standard;  or 

o  the  first  article  will  become  a  manufacturing  standard. 


First  Article  Tests  are  not  normally  employed: 

o  in  an  RAD  contract; 

o  if  the  item  is  on  the  Qualified  Products 
List  (QPL); 

o  for  commercial  products;  or 

o  products  with  complete  technical  specifications 
unless  the  requirements  are  so  novel  that  a  first 
article  examination  is  essential  to  assurance 
of  conforming  material . 

There  are  two  principal  cases  of  First  Article  Testing:  (1)  when 
the  contractor  will  perform  the  tests,  and  (2)  when  the  government  will 
perform  the  tests.  Combination  testing  may  be  desirable  also  with  each 
party  performing  some  of  the  tests.  There  is  a  standard  provision  avail¬ 
able  (DAR  7-104. 55(i)  that,  when  used,  makes  it  mandatory  that  the  production 
items  and  the  First  Article  be  manufactured  in  the  same  facility.  If  it 
is  necessary  for  personnel  from  the  purchasing  and/or  technical  organizations 
to  evaluate  the  First  Article  item,  care  should  be  taken  to  assure  the 
item  does,  in  fact,  conform  before  the  demonstration.  There  are  major 
quality  considerations  in  the  First  Article  process  to  be  examined,  both 
covering  the  requirements  and  the  conformance  to  the  requirements  as  well. 
First  Article  Testing  require  a  significant  degree  of  planning  and  other 
pre-testing  activities,  particularly  when  major  systems  are  involved. 

DAR  1-1900  should  be  reviewed  for  contracting  criteria  and  a  major  technical 
review  is  required  to  assure  that  the  test  will,  in  fact,  demonstrate  that 
the  requirements  have  been  achieved. 

Clauses  for  Preservation,  Packaging  and  Packing  Requirements,  DAR  7-104.67 

These  requirements  are  an  essential  part  of  a  supply  contract  and 
products  can  be  rendered  unusuable  as  a  consequence  of  inattention  to  them. 

It  is  essential  that  quality  personnel  in  purchasing  and  technical  activities 
determine  what  protection  will  be  required  and  that  contract  administration 
personnel  assure  that  the  requirements  are  achieved.  Instructions  in  these 
matters  for  personnel  in  contracting  offices  are  included  in  DAR  1-1204. 
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Warranty  Clauses,  PAR  7-105.7 

A  warranty  is  a  promise  given  by  a  seller  to  a  buyer  regarding  the 
nature,  usefulness  or  condition  of  a  product  or  service  to  be  furnished. 

The  actual  warranty  is  used  in  the  contract  to  establish  the  rights  and 
obligations  of  the  buyer  and  the  seller.  The  warranty  usually  provides 
that  for  some  period  after  acceptance,  the  product/service  will  perform 
as  originally  stated.  During  this  period,  the  government  may  require 
corrective  action  by  the  contractor  in  the  event  defects/non-conformance 
are  encountered.  If  there  was  a  government  design,  correction  action 
would  be  limited  insofar  as  requiring  the  contractor  to  take  action. 

However,  workmanship  and  material  and  conformance  are  the  responsibility 
of  the  contractor.  DAR  1-324  provides  extensive  guidance  for  personnel 
establishing  warranty  requirements  in  contracts.  Contract  administration 
personnel  must  be  alert  for  the  inclusion  of  such  clauses  and  must  plan 
their  actions  so  that  proper  investigations  are  possible  in  the  event 
warranty  claims  are  processed. 

Standards  of  Work,  PAR  7-302.3 

This  clause  simply  states  that  the  performance  of  work  and 
services  shall  conform  to  high  professional  standards.  When  the  clause 
appears  in  a  contract.  Government  quality  personnel  should  attempt  to 
relate  work  to  any  known  and  existing  defined  standard.  Some  definitions 
should  be  established  and  should  be  agreed  upon  by  the  firm  and  the 
Government.  Work  should  be  examined  for  adherence  to  the  standard. 

Mew  Material ,  PAR  7-104,48 

It  is  generally  expected  that  supplies  provided  under  contract 
will  be  new.  However,  there  may  be  circumstances  as  described  in  DAR  1-1208 
when  it  is  desirable  to  accept  items/materials  which  are  not  new.  Surplus 
material  may  also  be  a  matter  of  concern  and  its  utilization  is  described 
In  DAR  7-104.49.  Since  contracts  will  normally  include  these  clauses, 
quality  personnel  should  be  aware  of  them,  and,  in  the  event  that  other  than 
new  material  is  proposed,  should  review  the  materials/items  in  light  of  user 
needs  and  contract  terms  and  conditions. 
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Make  or  Buy  issues  may  have  an  impact  on  the  product  quality. 

When  the  clause  is  invoked  in  a  contract  the  preliminary  review  of  the 
program  should  consider  the  primary  concerns  of  price,  delivery  and  quality 
and  performance  of  the  products  involved.  Proposed  "make"  items  need  a 
specific  review  of  the  make  quality  as  opposed  to  what  the  quality  would  be 
if  the  product  were  purchased  elsewhere.  There  are  other  considerations, 
of  course,  such  as  tooling,  availability  of  personnel,  costs,  etc.  Product 
quality  aspects  of  the  two  situations  should  be  a  major  consideration. 

DAR  3-900  provides  further  guidance. 

Rights  in  Data  and  Computer  Software,  OAR  7-104.9 

Technical  data  encompasses  a  broad  field  of  documentation  in  a 
wide  variety  of  forms.  For  example:  drawings,  photographs,  written  texts, 
computer  printouts,  manuals,  computer  programs,  etc.  and  the  contract  clause 
may  take  several  forms.  One  major  issue  is  the  question  of  who  has  rights 
to  use  the  data  and  what  limitations  are  involved.  There  is  considerable 
guidance  existing  in  such  documents  as  AFCMPR  74-1,  AFSCR  310-1  and  the 
AFCriD  supplement  and  AFSCM  310-2.  In  view  of  the  involvement  of  quality 
personnel  in  configuration  control,  inspection  of  data,  etc.,  the  specific 
clauses  in  the  contract  should  be  reviewed  closely.  Definitions,  rights, 
subcontractor  relationships,  possible  deferred  delivery,  and  warranty  of 
data  are  examples  of  matters  that  are,  or  should  be,  contractually  defined. 
DAR  Section  9  provides  further  guidance  on  Patents  Data  and  Copyrights  in 
connection  with  the  procurement  of  supplies  and  services. 

Material  Inspection  and  Receiving  Report,  DAR  7-104.62 


In  most  instances,  this  clause  will  be  included  in  contracts 
where  it  is  anticipated  that  a  distinct  object  or  entity  will  be  delivered. 
The  clause  will  be  included  whether  items  are  priced  separately  or  not. 

The  contractor  must  prepare  a  Material  Inspection  and  Receiving  Report 
(MIRR)  in  accordance  with  instructions  in  DAR  Appendix  1.  The  form  will 
usually  be  the  DD  250  and  this  documents  the  official  procurement  quality 
assurance  act  of  acceptance  of  supplies,  services  and  shipments.  They 


also  have  other  uses  for  various  organizations  in  the  logistics  support 
activities.  The  contractor  is  most  concerned  with  its  use  as  part  of  the 
documentation  which  is  used  by  fiscal  authorities  as  justification  for 
payment.  The  requirements  for  government  approval  of  the  MIRR  (DD  250)  is 
an  effective  motivator  available  to  quality  assurance  personnel. 

Government  Property,  PAR  7-104.24 

The  controls  and  administration  required  for  government  property 
are  lengthy  and  complex,  e.g.,  DAR  Sections  XIII,  XXIV,  PAR  Appendices  B 
and  C  and  other  documents  as  well.  There  are  numerous  categories  of 
government  property  and  associated  instructions.  The  quality  role  will 
vary  from  contractor  to  contractor  and  any  unique  requirements  placed  for 
the  quality  assurance  activities  should  be  clearly  described.  Of  all  of 
the  categories  of  government  property,  the  one  which  normally  relates  closely 
to  the  quality  role  is  Military  Property  (DAR  13-101.7)  which  is  designed 
for  military  operations  and  includes  end  items,  integral  components  of 
systems  and  related  support  equipment.  In  the  event  MIL-Q-9858  is  in  the 
contract  and  Military  Property  is  a  part  of  the  contract,  the  quality 
assurance  role  is  at  least  in  part  defined.  Government  material  (DAR  T3-101 .4) 
is  a  quality  consideration  as  is  special  tooling  (DAR  13-101.5),  and  special 
test  equipment  (DAR  13-101.6).  This  brief  discussion  is  intended  only  to 
highlight  quality  considerations  when  government  property  is  involved  in 
a  contract  and  QA  personnel  must  evaluate  each  contract  requirement. 

Engineering  Change  Proposals,  DAR  7-104.89 

Engineering  Change  Proposals  (ECPs)  are  discussed  in  DAR  26-200  as 
a  part  of  the  general  subject  of  contract  modifications.  They  relate  to 
other  procedural  documents  such  as  the  Configuration  Management  System.  The 
essence  of  the  quality  assurance  role  is  to  assure  that  there  is  no  diminution 
of  performance  characteristics  that  are  essential  to  the  user.  It  is 
essential  that  documentation  of  approvals  be  confirmed  before  modified 
equipments  are  accepted  and  that  there  be  a  clear  equipment  oriented  audit 
trail  that  identifies  the  items  in  which  the  changes  appear  as  opposed  to 
those  manufactured  prior  to  the  change. 
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Inspection  of  Supplies  and  Correction  of  Defects,  PAR  7-203.5 

This  clause  must  be  incorporated  into  all  cost  reimbursement 
type  supply  contracts.  It  does  not  incorporate  minimum  quality  systems 
as  does  the  inspection  clause  DAR  7-103.5.  If  a  system  is  required, 

MI L- I -45208  or  MIL-Q-9858  or  some  specifically  tailored  or  designed  system 
may  be  included  in  the  contract.  This  is  a  penalty-oriented  motivational 
clause  in  that  if  defective  supplies  are  delivered  and  not  replaced  with 
acceptable  material,  there  are  monetary  penalties.  These  penalties  may 
result  in  a  reduction  of  any  fixed  fee  or  even  return  of  fee  paid.  It  also 
may  result  in  the  government  contracting  elsewhere  for  the  supplies  and 
charging  to  the  firm  increased  costs  that  occur.  This  clause  is  more 
complex  than  described  above.  However,  it  can  have  a  serious  impact  on 
a  firm.  Government  and  contractor  personnel  involved  in  the  contract's 
execution  and  administration  should  recognize  its  presence  and  assure  that 
there  is  no  cause  for  assessing  possible  penalties. 

C.  CLAUSES  INDIRECTLY  RELATED  TO  THE  QUALITY  ASSURANCE  FUNCTION 

Title  and  Risk  of  Loss,  DAR  7-103.6 

The  importance  of  this  clause  resides  in  the  fact  that  upon 
formal  acceptance,  title  of  material  is  transferred  to  the  government 
regardless  of  when  or  where  possession  is  taken.  The  acceptance,  delivery 
times  and  locations  stated  in  the  contract  provisions  must  be  thoroughly 
reviewed  to  determine  when  the  responsibility  for  risk  of  loss  or  damage 
passes  to  the  government.  It  is  not  necessarily  synonymous  with  acceptance. 

Liquidated  Damages,  DAR  7-105.5 

The  appearance  of  this  clause  is  an  indication  that  the  delivery 
schedule  is  of  such  importance  that  if  delays  are  encountered  the  government 
will  suffer  damages.  If  delivery  is  delayed,  a  certain  amount  of  charges 
will  be  assessed  each  day  until  delivery  occurs.  This  clause  is  a  signal 
that  delivery  is  of  unusual  importance  and  quality  planning  will  have  a 
greater  than  normal  urgency  or  importance.  Quality  personnel  may  not  be 
aware  of  the  reason  for  this  clause  in  the  contract  or  be  involved  in  the 
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consequences.  However,  non-conformances,  i.e.,  quality  defects,  may  lead 
to  assessment  of  the  charges.  Therefore,  a  continuing  review  of  progress 
and  difficulties,  particularly  as  they  relate  to  product  quality  and 
performance  is  advisable.  DAR  1-310  provides  general  policies  and 
procedures  relating  to  this  clause. 

Management  Systems,  PAR  7-104,50 

Management  Systems  are  not  a  product  in  the  hardware  sense. 

They  are,  in  fact,  a  service  which  has  been  contracted  for  as  part  of  a 
complete  contract.  A  system  is  described  in  OAR  1-331  as  a  documented 
method  for  assisting  managers  in  achieving  a  management  objective. 

MIL-Q-9858,  Quality  Program,  is  an  example  of  a  management  system  as 
is  MIL-STD  785,  Reliability  Program.  Normal ly,  the  organizational 
elements  that  established  the  management  system  requirement  will  evaluate 
accomplishments.  The  systems  should  be  itemized  on  the  form  DD  1660  and 
there  should  be  some  assignment  of  responsibility  for  each,  thus  assuring 
conformance  with  the  system  requirements. 

Value  Engineering,  GAP.  7-104.4 

Value  Engineering  (VE)  is  an  incentive  technique  to  reward  firms 
that  can  reduce  the  cost  of  items  or  services  without  negatively  impacting 
"needed  performance,  quality,  maintainability,  reliability,  or  interchange- 
ability"  (DAR  1-1701).  VE  is  not  limited  to  production  contracts  but  may 
b?  applied  to  any  of  the  life  cycle  phases.  There  may  also  be  a  requirement 
for  a  VE  program  or  management  system.  VE  is  included  here  due  to  occasional 
misgivings  that  VE  changes  "reduce"  the  "quality"  of  the  product.  Quality 
assurance  personnel  who  often  are  concerned  with  VECP  changes  must  consider 
the  proposal  in  terms  of  user's  needs  as  opposed  to  arbitrary  concerns  about 
what  "quality"  may  be  affected.  VECPs  may  properly  originate  in  quality  areas 
such  as  those  where  an  outdated,  inefficient  test  or  inspection  method  has  been 
specified.  The  quality  assurance  organization  at  the  technical  activity  which 
originated  the  requirement  should  participate  in  the  review  of  VECP  recommendations 
from  contracting  organizations. 
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Progress  Payments,  PAR  7-104.35 


Progress  payments  are  authorized  by  the  inclusion  of  this  clause 
and  payments  are  generally  based  on  requests  from  contractors  which  relate 
to  work  progress.  They  are  not  normally  directly  related  to  the  quality 
function  other  than  as  a  part  of  the  acceptance  process.  Among  the  conditions 
under  which  progress  payments  are  reduced  or  suspended  may  be  the  situation 
in  which  the  contractor  is  not  progressing  satisfactorily  in  the  agreed 
upon  work.  The  quality  assurance  routine  observation  of  conditions  may 
provide  the  contracting  officer  with  necessary  information  which,  when  made 
known,  will  result  in  consideration  of  changes  to  progress  payment  schedules. 

Payments,  PAR  7-103.7 

This  clause  establishes  in  contractual  language  the  fact  that 
the  contractor  shall  be  paid  upon  the  submission  of  invoices  or  vouchers 
for  supplies  delivered  and  "accepted."  The  title  transfer,  inspection 
and  delivery  of  material  predominates  the  quality  role  in  contract  proceedings. 
The  DD  250  normally  is  submitted  as  part  of  the  documentation  for  the 
paying  office.  It  is  required  in  fixed  price  supply  and  RSD  contracts. 

Aircraft,  Missile  and  Space  Vehicle  Accident  Reporting  and  Investigation, 

PAR  7-104.81 


This  clause  is  included  at  the  discretion  of  the  contracting 
officer.  It  is  included  due  to  the  fact  that  accidents  may  be  the  result 
of  defective  parts,  workmanship  or  design.  Quality  of  material  will  likely 
be  an  issue  unless  the  cause  is  clearly  operator  error. 

Order  of  Precedence,  PAR  7-204.40 


This  clause  is  used  at  the  discretion  of  contracting  officials 
when  preparing  solicitations.  It  is  intended  to  preclude  conflicting 
provisions.  If  it  is  incorporated  into  the  contract,  it  can  become  a  referee 
document  when  questions  of  precedence  within  contract  documentation  occurs. 

It  is  a  standard  clause  and  should  be  included.  Quality  personnel  should 
assure  its  inclusion  in  complex  contracts  and  recognize  it  in  the  Contract 
Administration  function. 
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Production  progress  reporting  is  included  here  only  because 
quality  personnel  may  be  requested  to  aid  in  determining  production  status. 
Quality  personnel  are  also  in  a  position  where  they  may  observe  situations 
that  can  have  a  negative  effect  on  anticipated  production  rates.  DAR  25-200 
provides  comprehensive  information  on  the  production  reporting  procedures. 


Safety  is  included  here  due  to  the  frequent  involvement  of 
quality  personnel  in  such  functions  as  inspection  of  explosive  material 
storage  sites,  explosive  material  transportation  vehicles,  equipment  tests, 
etc.  DoD  Handbook  4125. 26M  is  used  as  a  contractor's  manual  for  such 
material.  DAR  1-323  provides  contracting  guidance  regarding  the  utilization 
of  the  clause. 


This  guidebook  has  provided  a  broad  overview  of  the  project 
management  process.  It  has  provided  a  view  of  the  contract  language 
that  may  appear  in  USAF  contracts  and  provides  some  specific  suggestions 
for  monitoring  system  quality  status  and  progress. 

Throughout  the  life  cycle  there  are  many  and  diverse  technical 
and  administrative  requirements  that  impinge  on  the  quality  assurance  role 
of  assuring  product  conformance  with  requirements.  It  is  essential  that 
the  organizational  interfaces  mesh  smoothly  and  ultimately  result  in  the 
production  of  which  is  commonly  called  a  "quality  product."  It  is  not 
the  quality  assurance  responsibility  to  assume  this  management  role  but 
rather  the  project  manager  that  must  recognize  the  role  of  all  of  the 
organizational  elements  and  their  contribution  to  overall  product 
quality.  Figure  1-3  is  a  visual  illustration  of  the  technical  disciplines 
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PRODUCT  QUALITY  DESIGN  REVIEW  PROCEDURES 

The  purpose  of  this  Appendix  is  to  provide  a  procedure  by  which 
the  adequacy  and  quality  of  designs  may  be  evaluated.  In  the  absence  of 
other  instructions,  it  will  provide  a  common  baseline  for  preparing  for, 
and  participating  in,  a  design  review  and  will  assure  the  performance  of 
an  objective  review.  In  order  to  effectively  participate  in  the  review, 
the  participant  should  obtain  material  similar  to  that  in  Part  A  of  each 
section  in  time  to  assure  understanding  of  the  system  issues.  It  is 
assumed  that  there  will  be  a  review  board  chairman  who  will  provide  a  list 
of  attendees,  completed  actions  and  open  items  for  further  study. 

This  appendix  is  organized  as  follows: 

I.  System/Subsystems  Design  Review 

A.  Required  Information 

B.  Checklist 

II.  Circuit/Hardware  Design  Review 

A.  Required  Information 

B.  Checklist 

III.  Test  Procedure  Review 

A.  Required  Information 

B.  Checklist 

IV.  Computer  Software  Design  Review 

A.  Required  Information 

B.  Checklist 
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SYSTEM/SUBSYSTEM  DESIGN  REVIEW 
A.  REQUIRED  INFORMATION 

1.  A  block  diagram  in  which  delineation  is  carried  down 
to  the  subsystem  and/or  functional  component  level. 

2.  Design  specifications. 

3.  A  description  of  the  intended  function. 

4.  Identification  of  risk  areas  and  design  margin. 

5.  Alternate  approaches  to  the  design  which  were  considered. 

6.  Why  the  selected  approach  was  chosen. 

7.  Supplemental  information  that,  in  the  designer's  opinion, 
will  facilitate  understanding  of  requirements  and/or 
approach. 

8.  Reliability  and  maintainabil i ty  allocations. 

9.  Unusual  design  requirements,  e.g,  tight  electrical  or 
physical  requirement,  demonstration  testing,  combatibil ity 
with  other  systems,  etc. 

10.  Maintenance  concept. 

11.  System  partitioning  and  mechanical  packaging. 

12.  Subsystem/system  test  results. 

13.  Performance  and  interface  requirements  upon  which  design 
is  based. 
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CHECKLIST  FOR  SYSTEM/SUBSYSTEM  DESIGN  REVIEW 

1.  Does  the  design  meet  the  mission  requirements  and 
specifications? 

2.  What  alternate  approaches  were  considered? 

3.  What  are  the  limits  of  satisfactory  performance? 

4.  Has  a  system/subsystem  test  plan  been  identified? 

5.  What  problems  areas  and  failure  modes  can  be  anticipated 
based  on  previous  experience  and/or  analysis,  and  should 
they  receive  further  study? 

6.  Have  reasonable  performance/reliability/maintainability 
tradeoffs  been  made?  Define  them. 

7.  Will  redundancy  be  required  to  meet  reliability  goals, 
and  if  so,  how  can  it  be  done? 

8.  Are  interfaces  properly  identified  and  specified? 

9.  Have  all  environmental  factors  been  considered? 

10.  Can  environment  be  limited  to  a  smaller  range? 

11.  What  special  test  equipment  or  facilities  will  be  required 
to  checkout  system,  subsystem  or  components? 

12.  Is  the  design  cost  effective? 

13.  Have  operating  life,  storage  life,  and  shelf  life  been 
considered? 

14.  Has  a  tolerance  study  been  completed  and  an  optimum  error 
budget  developed?  Is  it  optimum  and  realistic  for  subsequent 
detailed  design? 

15.  Have  necessary  safety  considerations  been  implemented  in  design? 

16.  Has  a  maintainability  concept  been  identified? 

17.  Have  risk  areas  been  identified?  What  are  they? 

18.  What  physical  limitations  exist? 

19.  Have  all  "purchased  items"  used  in  system/subsystem  been 
defined? 

20.  Have  all  documentation  requirements  been  defined? 

21.  Have  human  factors  been  considered? 
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II.  CIRCUIT/HARDWARE  DESIGN  REVIEW 
A.  REQUIRED  INFORMATION 

1.  Performance  requirements  for  the  circuit/hardware  item 
to  be  reviewed. 

2.  Any  assumptions  made  in  identifying  design  baseline. 

3.  Functional  block  diagram  showing  relation  of  item(s)  to 
be  reviewed. 

4.  Schematics  and  parts  list  of  each  item.  Parts  list  should 
show  part  stress  level  that  exists  in  design. 

5.  Is  there  any  part  or  material  for  which  a  waiver  will  be 
required. 

6.  Summary  of  reliability  estimate  and  allocation,  resulting 
MTBF  and  failure  rate  base. 

7.  Identification  of  risk  areas. 

8.  Breadboard  test  results  (when  available). 

9.  Supplemental  information  which  will  facilitate  understanding 
of  requirements,  approach  or  results. 

10.  Circuit  failure  mode  analysis  (as  applicable). 
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CHECKLIST  FOR  CIRCUIT/HARDWARE  DESIGN  REVIEW 

1.  Ooes  the  design  meet  all  performance  requirements? 

2.  What  key  performance  requirements  have  created  the  greatest 
design  complexity? 

3.  If  any  of  the  previous  factors  were  relaxed  or  if  redistribution 
of  tolerances  were  possible,  what  savings  would  result? 

4.  What  alternate  approaches  were  considered? 

5.  Is  the  analysis  of  the  design  sufficiently  complete  and 
accurate? 

6.  Have  off-the-shelf  or  standard  items  been  used  where 
practical ? 

7.  What  problem  areas  and  failure  modes  can  be  anticipated 
based  on  previous  experience  and  analysis?  Should  they 
receive  further  study? 

8.  What  parts  are  critical  to:  (1)  circuit/hardware  performance; 

(2)  system/subsystem  performance? 

9.  Is  the  design  cost  effective? 

10.  Coes  the  item  require  special  handling?  Is  it  designed  for 
ease  of  fabrication,  test,  assembly  and  maintenance? 

11.  Has  use  of  adjustments  been  minimized? 

12.  Has  a  complete  statistical  or  worst  case  analysis  been 
performed? 

13.  Is  the  design  physically  and  electrically  compatible  with 

other  parameters?  (Dimensions,  input  and  output  characteristics, 
noise  generation  and/or  susceptibility.) 

14.  Do  breadboard  test  results  agree  with  analysis? 

15.  Has  minimum  use  of  tight  tolerance,  precision  parts  or 
parameters  been  made? 

16.  Are  any  special  test  equipment  or  techniques  required? 

17.  Are  there  any  parts  requiring  waivers  or  qualification? 

18.  Are  power  supplies  properly  decoupled? 

19.  Have  critical  layout  and  path  length  restrictions  been 
identified? 
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B.  CHECKLIST  FOR  CIRCUIT/ HARDWARE  DESIGN  REVIEW  (Cont’d) 

20.  Has  sneak  circuit  analysis  been  performed? 

21.  Have  risk  areas  been  defined?  What  are  they? 

22.  Has  a  fail-safe  design  philosophy  been  used? 

23.  Is  protection  against  secondary  failures  incorporated 
where  possible? 

24.  Have  human  factors  been  considered?  (Man-machine  interface, 
human  safety  factors) 

I  25.  Are  multiple  sources  available  for  each  part  type? 

I 

j  26.  Is  the  interaction  between  adjustments  and  other  circuit 

i  parameters  minimized? 

27.  Have  all  parts  been  applied  within  company/military  standard 
derating  limits? 

t 

■  28.  Are  special  QC  or  vendor  problems  anticipated? 

I 

t 

1 


i 
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III.  TEST  PROCEDURE  REVIEW 

A.  REQUIRED  INFORMATION 

1.  Proposal,  performance  requirements,  interface  requirements, 
and  test  requirements  upon  which  the  test  procedure  is  based. 

2.  Test  flow  diagram. 

3.  Description  of  tests  to  be  performed. 

d.  Identification  of  risk  areas. 

5.  Alternate  test  methods  considered. 

6.  Supplemental  information  which  the  test  engineer  feels  will 
facilitate  understanding  the  requirements  and/or  approach. 

7.  Test  philosophy. 

8.  Reason  for  test  requirement  change. 

9.  Test  equipment  selection  and  tradeoff  criteria. 

10.  Identification  of  measurement  tolerances  and  acceptance 
criteria. 
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B.  CHECKLIST  FOR  TEST  PROCEDURE  REVIEW 

1.  Does  the  test  plan  encompass  an  integrated  test  cycle  from 
system  level  down  to  the  lowest  test  level? 

2.  Has  a  requirement  been  identified  for  each  test? 

3.  Is  the  test  feasible? 

4.  Has  all  test  equipment,  both  special  purpose  and  commercial 
been  identified  to  perform  the  tests? 

5.  For  tests  which  use  commercial  test  equipment,  are  the  test 
requirements  within  the  capability  of  the  equipment? 

6.  Are  tests  performed  in  a  logical  sequence? 

7.  Do  design  changes  need  to  be  made  to  facilitate  or  improve 
testing? 

8.  Do  the  test  procedures  verify  interface  compatibility? 

9.  Are  error  tolerances  and  compatibility  verified  at  the 
lowest  test  level? 

10.  Are  all  self-test  loops  exercised  for  both  GO  and  NO  GO  paths? 

11.  Are  there  any  areas  of  over  or  under  testing? 

12.  Are  in-plant  test  facilities  adequate  to  perform  all  testing? 

13.  Is  the  test  equipment  available? 

14.  Has  a  measurement  error  analysis  been  performed? 

15.  Has  consideration  been  given  to  verification  of  requirements 
by  analysis  and  inspection  as  well  as  testing? 

16.  Have  all  supporting  (Doth  equipment  and  personnel)  resources 
been  identified? 

17.  Have  the  responsibilities  of  the  test  participants  been 
identified? 

18.  Are  any  operational  tests  required  to  verify  requirements? 

19.  Are  the  tests  organized  to  maximize  use  of  early  test  results 
in  later  test  phases? 

20.  Has  the  criteria  for  success,  for  failure  been  identified? 

21.  Has  sufficient  consideration  been  given  for  data  storage 
and/or  reduction? 


1-8 


CHECKLIST  FOR  TEST  PROCEDURE  REVIEW  (Cont'd) 

22.  Are  any  GFE  equipment  or  facilities  required? 

23.  Has  adequate  test  tolerance  degradation  been  provided 
for  test  equipment,  cabling,  connector  interfaces,  etc.? 

24.  Is  the  test  schedule  compatible  with  the  design  cycle? 

25.  Has  consideration  been  given  to  the  test  data  being  used 
for  reliability,  maintainability  and  safety  analysis? 

26.  Have  risk  areas  been  defined?  What  are  they? 
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IV.  COMPUTER  SOFTWARE  DESIGN  REVIEW 


REQUIRED  INFORMATION 


1.  Specification  and  interface  requirements  upon  which  program 
design  is  based. 

2.  Structured  flow  in  which  the  program  delineation  is  carried 
down  to  the  subprogram  and/or  routine/subroutine  level. 

3.  A  description  of  the  intended  function,  with  benchmark 
definitions  for  comparative  analysis. 

4.  Identification  of  risk  areas. 


5.  Alternate  approaches  to  the  design  which  were  considered. 

6.  Why  the  selected  approach  is  best. 

7.  Supplemental  information  that  in  the  programmer's  opinion 
will  facilitate  understanding  of  requirements  and/or 
approach . 

8.  Input/output  data  formats. 

9.  Program  user  manual  (if  applicable). 

10.  Tradeoff  indicating  machine  more  effective  for  program 
util ization. 


11 .  Test  results  . 
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CHECKLIST  FOR  COMPUTER  SOFTWARE  REVIEW 

1.  Does  the  program  meet  requirements  or  specifications? 

2.  Is  the  program  organization  logical? 

3.  Were  alternate  approaches  considered? 

4.  Are  there  any  other  approaches  which  should  be  considered? 

5.  Is  analysis  of  the  program  sufficiently  complete?  Do  any 
areas  require  additional  study? 

6.  What  are  the  limits  of  satisfactory  performance? 

7.  Are  intercommunications  requirements  betv/een  sub-programs, 
routines  and  sub-routines  properly  identified  and  specified? 

8.  Is  program/sub-program  partitioning  considered  optimum? 

9.  What  limitations  exist? 

a.  Memory  size 

b.  Cycle  time  for  iterative  programs. 

10.  Is  an  error  budget  required?  If  so,  has  the  error  analysis 
been  completed?  Is  is  optimum  and  realistic  for  subsequent 
detailed  development? 

11.  Have  reasonable  performance/memory  size  tradeoffs  been  made? 

12.  Have  reasonable  performance/loop  cycle  time  tradeoffs  been  made? 

13.  Can  existing  programs,  sub-programs  or  routines  be  modified 
for  use  in  any  functional  program  requirements? 

14.  Has  a  debugging  plan  been  identified? 

15.  What  special  test  programs  will  be  required  to  check  out  system? 

16.  Has  a  suitable  set  of  check  solutions  or  simulations  been  made? 

17.  Is  the  program  user  manual  complete?  Does  it  adequately  specify 
operating  limitations,  interpretation  of  output  data,  and  input 
data  required? 

18.  Are  input/output  formats  defined? 

19.  Have  diagnostic  statements  been  included  to  detect  input  errors? 

20.  What  computer  is  recommended  for  running  the  program?  What  is  it? 
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Characteristics  are  examples  for  illustrative  purposes  only. 
Periods  are  monthly  but  should  be  established  as  appropriate. 


PRODUCT  QUALITY  RELATED  CLAUSES 


